You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a first-person record of how I took Karpathy's LLM-Wiki intuition and turned it, step by step, into the llm-skill project. I'll walk through: where I started, why I chose this particular layering, what each layer solves.
1. Starting point: what I actually took away from LLM-Wiki
A single llm-wiki is good. But you will eventually want more than one. And then you'll want them to talk to each other. That's what this is.
federated-llm-wiki should let you deploy vaults in minutes, connect them in a clean DAG, plug in agents and UI components, and scale from a single personal wiki to a network of federated knowledge bases — covering completely unrelated topics — under one roof.
Not centralized — So, no global authority. Not decentralized — So, no trustless chaos. Federated: each vault governs its own knowledge; connections are explicit, directional, and acyclic.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
LLM-Wiki Obsidian Setup Guide — full implementation of Karpathy's llm-wiki pattern
Setting Up the LLM-Wiki Pattern in Obsidian
Based on Karpathy's llm-wiki pattern — using LLMs to incrementally build and maintain a persistent, interlinked wiki from raw sources rather than re-deriving knowledge on every query.
Quick Start — Paste This Into Your Agent
Copy the prompt below into Claude Code, Codex, or any LLM agent with file access. It will scaffold the wiki, configure the tooling, and update your CLAUDE.md with the schema. Then you just start ingesting sources.
I want to set up the llm-wiki pattern in my Obsidian vault. Follow this guide exactly:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A slightly modified implementation of Ken Perlin's improved noise that allows for tiling the noise arbitrarily.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Performance comparisons of flex attention to SDPA with sample packing
TLDR: We can nearly 7x (+560%) our throughput by turning compile + sample packing + flex attention on. Compared to packing with compiled SDPA, compiled flex attention boosts throughput by over 2x (140%) for Llama3 8B at max sequence length of 8192. This effect is more pronounced as sequence length increases as it scales more efficiently.
Flex attention introduces a mask_mod argument that enables flash attention with arbitrary masks. This also removes the need
to hold a mask in memory before computing attention, which can get expensive with longer sequence lengths. Here we benchmark
the throughput and memory usage of sample packing using flex attention and sample packing using standard SDPA.